Knowledge-Free Induction of Morphology Using Latent Semantic Analysis

Morphology induction is a subproblem of important tasks like automatic learning of machine-readable dictionaries and grammar induction. Previous morphology induction approaches have relied solely on statistics of hypothesized stems and affixes to choose which affixes to consider legitimate. Relying on stemand-affix statistics rather than semantic knowledge leads to a number of problems, such as the inappropriate use of valid affixes ("ally" stemming to "all"). We introduce a semantic-based algorithm for learning morphology which only proposes affixes when the stem and stem-plusaffix are sufficiently similar semantically. We implement our approach using Latent Semantic Analysis and show that our semantics-only approach provides morphology induction results that rival a current state-of-the-art system. 1 I n t r o d u c t i o n Computational morphological analyzers have existed in various languages for years and it has been said that "the quest for an efficient method for the analysis and generation of word-forms is no longer an academic research topic" (Karlsson and Karttunen, 1997). However, development of these analyzers typically begins with human intervention requiring time spans from days to weeks. If it were possible to build such analyzers automatically without human knowledge, significant development time could be saved. On a larger scale, consider the task of inducing machine-readable dictionaries (MRDs) using no human-provided information ("knowledge-free"). In building an MRD, "simply expanding the dictionary to encompass every word one is ever likely to encounter...fails to take advantage of regularities" (Sproat, 1992, p. xiii). Hence, automatic morphological analysis is also critical for selecting appropriate and non-redundant MRD headwords. For the reasons expressed above, we are interested in knowledge-free morphology induction. Thus, in this paper, we show how to automatically induce morphological relationships between words. Previous morphology induction approaches (Goldsmith, 1997, 2000; D4Jean, 1998; Gaussier, 1999) have focused on inflectional languages and have used statistics of hypothesized stems and affixes to choose which affixes to consider legitimate. Several problems can arise using only stem-and-affix statistics: (1) valid affixes may be applied inappropriately ("ally" stemming to "all"), (2) morphological ambiguity may arise ("rating" conflating with "rat" instead of "rate"), and (3) non-productive affixes may get accidentally pruned (the relationship between "dirty" and "dirt" may be lost)3 Some of these problems could be resolved if one could incorporate word semantics. For instance, "all" is not semantically similar to "ally," so with knowledge of semantics, an algorithm could avoid conflating these two words. To maintain the "knowledge-free" paradigm, such semantics would need to be automatically induced. Latent Semantic Analysis (LSA) (Deerwester, et al., 1990); Landauer, et al., 1998) is a technique which automatically identifies semantic information from a corpus. We here show that incorporating LSA-based semantics alone into the morphology-induction process can provide results that rival a state-oh the-art system based on stem-and-affix statistics (Goldsmith's Linguistica). 1Error examples are from Goldsmith 's Linguistica